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EXTENDED REPORT 

Evaluation of the RABBIT Risk Score for serious 
infections 

A Zink/'^ B Manger,^ J Kaufmann,^ C Eisterhues,^ A Krause,^ J Listing/ 
A Strangfeld^ 



ABSTRACT 

Objective To evaluate the Rheumatoid Arthritis 
Observation of Biologic Therapy (RABBIT) Risk Score 
for serious infections in patients with rheumatoid 
arthritis (RA). 

Methods The RABBIT Risk Score for serious infections 
was developed in 201 1 on a cohort of RA patients 
enrolled in the German biologies register RABBIT 
between 2001 and 2007. To evaluate this score, we 
used data from patients enrolled in RABBIT after 1 
January 2009. Expected numbers of serious infections 
and expected numbers of patients with at least one 
serious infection per year were calculated by means of 
the RABBIT Risk Score and compared with observed 
numbers in the evaluation sample. 
Results The evaluation of the score in an independent 
cohort of 1 522 RA patients treated with tumour necrosis 
factor a (TNFa) inhibitors and 1468 patients treated 
with non-biological disease-modifying antirheumatic 
drugs (DMARDs) showed excellent agreement between 
observed and expected rates of serious infections. For 
patients exposed to TNF inhibitors, expected as well as 
observed numbers of serious infections were 3.0 per 
100 patient-years (PY). For patients on non-biological 
DMARDs the expected and observed numbers were 
1.5/100 PY and 1.8/100 PY, respectively. The score was 
highly predictive in groups of patients with low as well 
as with high infection risk. 
Conclusions The RABBIT Risk Score is a reliable 
instrument which determines the risk of serious infection 
in individual patients based on clinical and treatment 
information. It helps the rheumatologist to balance 
benefits and risks of treatment, to avoid high-risk 
treatment combinations and thus to make informed 
clinical decisions. 



INTRODUCTION 

Serious infections are a major concern in patients 
treated with cytokine inhibitors or other biological 
agents. Randomised clinical trials as well as obser- 
vational cohort studies have shown that there is an 
increased risk of serious infection in patients 
treated with tumour necrosis factor a (TNFa) inhi- 
bitors. The risk, however, seems to decrease over 
time: at the start of treatment, an up to 4.5-fold 
risk was reported,^ whereas after 1 year of treat- 
ment no increase in risk over conventional therapy 
was seen in observational cohort studies.^ ^ ^ When 
interpreting data from observational studies, it is 
important to distinguish the changes in overall risk 
in the cohorts from the changing risks occurring in 
individual patients. We showed previously that the 



decline in risk over time observed in cohorts can be 
attributed to (i) the increased loss to follow-up of 
patients with higher susceptibility to infections, as 
well as (ii) a risk decline in individual patients 
attributable to improvement in clinical status and 
reduced glucocorticoid (GC) intake. Taking base- 
line characteristics and time-dependent changes in 
the clinical status and treatment of individual 
patients into account, we were able to calculate the 
expected infection risk for specific patient profiles 
and for individual therapies at any time point 
during the course of treatment.^ This risk estimate 
allows rheumatologists to align their therapy to the 
expected risk of individual patients. 

However, the risk calculation was based on one 
single cohort. The aim of this analysis was to evalu- 
ate the score with a new cohort of patients not 
included in its development. 

We used data from the German biologies register 
RABBIT, which is the German acronym for 
'Rheumatoid Arthritis (RA) Observation of Biologic 
Therapy'. 

METHODS 

The RABBIT Risk Score was developed on patients 
enrolled in RABBIT before 1 January 2007.^ For the 
evaluation of the risk score, we used an independent 
cohort of patients who were enrolled in RABBIT 
between January 2009 and January 2012 at start of 
treatment with a TNFa inhibitor or a non-biological 
disease-modifying antirheumatic drug (nbDMARD). 
Using the RABBIT Risk Score, we calculated for 
each patient the expected likelihood of a serious 
infection based on the individual risk profile. 

In our previous analysis we had estimated the 
number of serious infections per 100 patient-years by 
means of generalised estimation equations with a 
Poisson link function. The Poisson link is appropriate 
for counts such as numbers of infections. A more 
intuitive measure, however, is the percentage of 
patients with at least one infection per year. We there- 
fore calculated in addition a modified version of the 
risk score using the development sample of patients 
enrolled before 1 January 2007. A complementary 
log-log link function was applied to estimate the 
probability of at least one serious infection per year 
per patient. Both versions of the RABBIT Risk Score 
contain the following risk factors which had proven 
predictive of the infection risk in our previous ana- 
lysis: age (up to or above 60), functional status 
(assessed by the Hannover Functional Status 
Questionnaire, Funktionsfragebogen Hannover, 
FFbH), specific co-morbidities (chronic renal or lung 
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Table 1 Calculation of the RABBIT Risk Score 



Risk factors 




V.I I NumbGr of SGrious 
infections per 100 PYs 


V.2i Percentage of patients with 
at least one infection per year 


Intercept 


Always add 


-3.996 


-4.191 


Age 


If age >60 add 


0.479 


0.470 


Function (FFbH) 


Add 


-0.01014*FFbH 


-0.00865*FFbH 


Alternatively: HAQ 


Add 


0.362(HAQ-3.16) 


0.309(HAQ-3.16) 


Chronic lung disease 


If yes add 


0.522 


0.484 


Chronic renal disease 


If yes add 


0.441 


0.415 


Previous serious infection 


If yes add 


0.748 


0.992 


Number of treatment failures 


If >5 add 


0.443 


0.397 


Mean glucocorticoid dose 


If 7.5-14 mg/day add 


0.756 


0.782 


Mean glucocorticoid dose 


If > 1 5 mg/day add 


1.554 


1.355 


Treatment with TNF inhibitor 


If yes (last 3 months) add 


0.593 


0.589 


Calculate the sum of the corresponding values 




Sumi 


Sum2 


Rabbit Risk Score 


Calculate 


100*e'"'"'' 


100*(1-e-^^"") 



FFbH, Hannover Functional Status Questionnaire, Funktionsfragebogen Hannover; HAQ, Health Assessment Questionnaire; PY, patient-years; RABBIT, Rheumatoid Arthritis Observation of 
Biologic Therapy; TNF, tumour necrosis factor. 



disease), treatment with GCs (<7.5 mg/day, 7.5-15 mg/day or 
above), number of previous DA/LARD treatment failures, previous 
serious infection, and current treatment w^ith a TNFa inhibitor or 
an nbDAdARD. The expected numbers of serious infections as 
v^ell as the expected numbers of patients w^ith at least one infection 
were calculated according to the two versions of the RABBIT Risk 
Score and then compared to the observed numbers in the cohort 
or within defined subgroups of patients. 

Table 1 shows the risk factors included in the score, their 
weights and the formula for the calculation of both versions of 
the score. Since the German instrument FFbH for the measure- 
ment of functional capacity can be transformed into Health 
Assessment Questionnaire (HAQ) scores using a formula pub- 
lished by Lautenschlager et alj the RABBIT Risk Score can also 
be calculated with HAQ values. 



To evaluate the agreement between observed and expected 
event rates, we applied the Hosmer-Lemshow test to subgroups 
of patients was well as to 10 groups established according to the 
expected risk (figure 1). 

RESULTS 

Data on 1522 RA patients treated with TNFa inhibitors and 1468 
patients treated with nbDMARDs were available. The median 
observation time was 1.6 years. Table 2 compares baseline charac- 
teristics of patients enrolled in the development cohort to those 
enrolled in the evaluation cohort. The differences in patient char- 
acteristics, co-medication and previous treatments between the ori- 
ginal sample and the evaluation cohort indicate the changes in 
treatment strategies for RA in Germany. Patients enrolled in the 
register between 2001 and 2007 had higher disease activity in 
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Table 2 Baseline characteristics of patients 





Original sample 




Evaluation sample 




Anti-TNF 


nbDMARD 


Anti-TNF 


nbDMARD 


N 


3271 


1773 


1522 


1468 


Female, n (%) 


2556 (78.1) 


1394 (78.6) 


1143 (75.1%) 


1106 (75.3%) 


Age 


53.8 (12.3) 


56.2 (11.5) 


55.8 (12.9) 


58.2 (12.5) 


Disease duration (years), median (IQR) 


9 (5, 1 6) 


6 (3, 12) 


7 (3, 14) 


4 (2, 9) 


Follow-up time (years), median (IQR) 


3.1 (2.1, 4.9) 


3.3 (2.5, 5.0) 


1.5 (0.6, 2.1) 


1.6 (0.9, 2.4) 


Rheumatoid factor positive, n (%) 


2624 (80.2) 


1271 (71.7) 


1095 (72.8) 


866 (59.0) 


DAS28 


5.7 (1.2) 


5.1 (1.3) 


5.1 (1.3) 


4.6 (1.3) 


FFbH 


57.0 (23.0) 


66.6 (21.5) 


64.8 (22.8) 


71.2 (21.8) 


^mnUnn no\/or n (^/n\ 

JIIIUIxIMU IICVCl, II \ /0/ 


1 077 (Al 0) 

1 \J£. 1 1 .\J/ 


JOJ \HJ.\J/ 


fifi7 (AA ^) 

UU/ y-tH.J) 




No. of previous DMARDs 


3.3 (1.3) 


1.8 (1.0) 


2.5 (1.1) 


1.6 (0.9) 


No. of previous biologicals 


0.23 (0.6) 


0.01 (0.1) 


0.24 (0.57) 


0.07 (0.34) 


Glucocorticoids 7.5-14 mg/day, n (%) 


1027 (31.4) 


386 (21.8) 


383 (25.2) 


171 (11.6) 


Glucocorticoids >1 5 mg/day, n (%) 


491 (15.0) 


147 (8.3) 


117 (7.7) 


50 (3.4) 


Chronic lung disease, n (%) 


246 (7.5) 


112 (6.3) 


111 (7.3) 


88 (6.0) 


Chronic renal disease, n (%) 


139 (4.3) 


31 (1.8) 


57 (3.8) 


27 (1.8) 



Values are means and SDs if not otherwise specified. 

; nbDMARD, non-biological disease-modifying antirheumatic drug; TNF, tumour necrosis factor. 



both the anti-TNF and the nbDMARD cohorts, they had lower 
functional capacity, more previous DMARD failures, were more 
often rheumatoid factor positive and were more often treated with 
GCs >7.5 mg/day than patients enrolled between 2009 and 2012. 
However, there was no difference in the prevalence of co-morbid 
chronic lung or renal disease. 

We applied the original RABBIT Risk Score based on the 
number of infections per 100 patient-years to the evaluation 
cohort (table 3, columns 4-7). According to the score, 69.6 
infections were expected under TNF inhibitors. With 69 infec- 
tions observed in the evaluation cohort (corresponding to a rate 
of 3.0 per 100 patient-years) the agreement with the estimate 
was high. This was also the case in subgroups of patients with 
specific risk factors where we found a high concordance 
between expected and observed rates (p = 0.32 for disagree- 
ment). When we appHed the second version of the score which 
gives the percentage of patients affected by at least one infection 
per year, thus including the first infection per patient only, we 



again found high agreement between numbers of observed and 
expected cases (p = 0.24 for disagreement). In the patient groups 
with higher risks, the observed numbers of infections were insig- 
nificantly higher than expected. 

The comparison of expected and observed event rates in 
deciles of risk scores using the Hosmer-Lemshow test resulted 
in a high agreement as shown in figure 1. The p values for dif- 
ference between observed and expected were 0.85 for all 
patients, 0.29 for patients exposed to TNF inhibitors and 0.49 
for those exposed to nbDMARDs. 

We developed an online risk score calculator which can be 
accessed via the RABBIT website (http://wvw.biologika-register. 
de/risk score). The following example shows how the score can 
be used in daily practice to balance risks from different treat- 
ments: if we consider a patient aged 65 with RA and chronic 
obstructive pulmonary disease, this patient's estimated risk 
under methotrexate and 7.5 mg/day GCs is 4.7%. If the GC 
dose is increased to 15 mg/day, the risk increases to 8.2%. 



Table 3 Expected and observed numbers and rates of serious infections for V.1 (per 100 patient-years) and V.2 (% of patients with at least 
one serious infection during 12 months) 

v.1: Number of serious infections per V.2: Number and percentage of patients affected 









100 PY (CI) 






by at least one serious infection per year (CI) 












Exp. rate/ 








Exp. per 






n 


PY 


Exp. n 


Obs. n 


100 PY 


Obs. rate/100 PY 


Exp. n 


Obs. n 


year (%) 


Obs. per year (%) 


TNFa inhibitor, no risk factor 


764 


1059 


16.8 


16 


1.6 


1.5 (0.9-2.5) 


15.3 


16 


1.4 


1.5 (0.9-2.5) 


nbDMARD, no risk factor 


632 


816 


7.0 


6 


0.9 


0.7 (0.3-1.6) 


6.4 


6 


0.8 


0.7 (0.3-1.6) 


TNFa inhibitor, >1 risk factor*, no GC 


635 


871 


28.9 


37 


3.3 


4.3 (3.0-5.9) 


25.4 


33 


2.9 


3.8 (2.6-5.3) 


nbDMARD, >1 risk factor*, no GC 


674 


939 


15.5 


19 


1.7 


2.0 (1.2-3.2) 


14.0 


19 


1.5 


2.0 (1.2-3.2) 


TNFa inhibitor + GC, no other risk factor 


225 


196 


8.6 


5 


4.4 


2.6 (0.8-6.0) 


7.4 


4 


3.8 


2.0 (0.6-5.2) 


nbDMARD + GC, no other risk factor 


128 


86 


1.9 


2 


2.2 


2.3 (0.3-8.4) 


1.7 


2 


2.0 


2.3 (0.3-8.4) 


TNFa inhibitor, >1 risk factor* + GC 


206 


160 


15.3 


11 


9.6 


6.9 (3.4-12.3) 


12.8 


11 


8.0 


6.9 (3.4-12.3) 


nbDMARD, >1 risk factor* + GC 


141 


96 


5.1 


8 


5.3 


8.3 (3.6-16.4) 


4.4 


8 


4.6 


8.3 (3.6-16.4) 


TNFa inhibitor total 


1830 


2286 


69.6 


69 


3.0 


3.0 (2.3-3.8) 


60.9 


64 


2.7 


2.8 (2.2-3.7) 


nbDMARD total 


1575 


1937 


29.6 


35 


1.5 


1.8 (1.2-2.5) 


26.6 


35 


1.4 


1.8 (1.2-2.5) 



Bold: The confidence intervals of the observed rates overlap with the expected rates. There is no other test for insignificant difference necessary or sensible. 

*At least one of: chronic lung disease, chronic renal disease, age above 60 years, previous serious infection, high number of DMARD failures. GC: treatment with glucocorticoids 

>7.5 mg/day prednisolone equivalent. 

exp, expected; obs, observed; nbDMARD, non-biological disease-modifying antirheumatic drug; PY, patient-years; TNF, tumour necrosis factor. 
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Adding a TNF inhibitor would further increase the risk to 
14.3%. However, if therapy is effective and the GC dose can be 
tapered down to 5 mg/day, the risk decHnes to 3.9%. 

The calculator also contains weights for the more recently 
licensed substances rituximab, tocilizumab and abatacept, based 
on data from 1343, 825 and 444 patients exposed to the 
respective agents since 2007. Using the formula for TNF inhibi- 
tors, we calculated the difference between expected and 
observed incidences of at least one infection per year. The 
observed incidences were very similar to those expected (factors 
compared to TNF inhibitors: 1.15 for tocilizumab, 0.92 for 
rituximab and 0.82 for abatacept). The weights for the newer 
substances need further evaluation in a replication cohort. 

DISCUSSION 

In daily clinical practice, choices have to be made between treat- 
ment strategies with different benefit-risk ratios. Scoring 
systems that enable calculation of absolute numbers of expected 
adverse events per year or per 100 patient-years help the treat- 
ing rheumatologist to make informed decisions, taking the 
overall risk resulting from treatments and their alternatives as 
well as specific risks resulting from the demographic and clinical 
situation of individual patients into account. 

Using data from the Rochester cohort, Doran et al^ showed in 
2002 that, among others, RA severity, functional status and 
co-morbidity were predictors of serious infection. This group 
recently developed a risk score for serious infection, based on 
584 patients enrolled in the Rochester cohort between 1955 
and 1994, and evaluated the score in 410 patients enrolled 
between 1995 and 2007.^ The contribution of specific risk 
factors such as age, previous infection, co-morbidity or GC dose 
was similar as in our data. They found an HR of 3.6 for more 
than 10 mg GCs/day while we found a fourfold increased risk 
for > 15 mg/day compared to < 7.5 mg/day. Each co-morbidity 
doubled the risk which is comparable to our data. A direct com- 
parison of the two scores is not possible since the Rochester 
score does not include treatment with biological or non- 
biological DMARDs and our score does not include erythrocyte 
sedimentation rate and extra-articular manifestations of RA. 

A second infection risk score was developed very recently by 
Curtis et al^^ using two administrative databases in the USA. 
Predictors of serious infection were, as in our data, older age, 
co-morbid conditions, higher dosages of GCs and previous 
serious infection. Diabetes mellitus was also associated with a 
moderately increased risk in this study. We did not include dia- 
betes into our model due to low predictive capacity. Diabetes as 
a co-morbid condition may play a greater role in the USA: in 
the Curtis study, its prevalence was >30% in the older age 
group and >10% in the younger one, whereas only 8% of our 
patients had diabetes. The risk score is also not fully comparable 
to ours since parameters of RA disease activity or severity were 
not available in the US study. On the other hand, the Curtis 
score includes a number of items such as cancer screening or 
treatments for other chronic diseases that are not available to us. 

A strength of our risk score is that it includes time-varying 
parameters of disease activity and disability which results in 
more precise risk estimates on the individual level than treat- 
ment and sociodemographic factors only. The limitation is that a 
risk score is not generalisable to populations with substantially 
different risk distributions. Therefore, we consider our risk 
score valid for the Caucasian population with moderate to 
severe RA in Germany and probably also in Western Europe. 
Re-evaluation of the score in other RA populations would be 
useful to test its reliability beyond the RABBIT register. 



Correction notice This article has been corrected since it was published Online 
First. The last row of table 1 has been corrected. 
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